Field-Level Encryption for FlatBuffers
AES-256 encryption with HD wallet key derivation, meeting federal data-at-rest requirements
Capabilities
Field-Level Encryption
Encrypt individual fields within FlatBuffer records. The binary structure stays intact so upstream code can read encrypted values without schema changes.
HD Wallet Key Derivation
Keys derived from BIP-39 seed phrases using HKDF-SHA256. Each field can have its own derived key for granular access control.
High Performance
Generate and encrypt 1 million records at 50+ MB/s using WebAssembly. Streaming export avoids memory accumulation.
Zero Server Dependency
All cryptographic operations run client-side in the browser. No data leaves your device unless you explicitly export it.
Cryptographic Implementation
Federal Compliance Standards
This implementation uses cryptographic algorithms approved for protecting federal data at rest:
CNSA 2.0
Commercial National Security Algorithm Suite 2.0 mandates AES-256 for all National Security Systems. Quantum-resistant algorithm support required by 2025.
View PDF →SP 800-111
Guide to Storage Encryption Technologies for End User Devices. Endorses AES-256 for data at rest with proper key management.
View Publication →140-3
Security Requirements for Cryptographic Modules. AES-256 is approved for all security levels (1-4) in federal cryptographic modules.
View Standard →STIGs
Security Technical Implementation Guides require FIPS 140-validated AES encryption for protecting CUI and classified information.
View STIG Library →hd-wallet-wasm with built-in OpenSSL FIPS support.
Use Cases
Defense & Intelligence
Field-level encryption for mission data with per-user key derivation and need-to-know access control.
Healthcare (HIPAA)
Encrypt PHI fields while leaving non-sensitive metadata readable for indexing and routing.
Financial Services
Protect PII and transaction data at rest with audit-ready key management from HD wallets.
Zero-Trust Architectures
Data remains encrypted until the authorized recipient decrypts with their derived key.
Public Key Infrastructure
Alice encrypts data with Bob's public key. Only Bob can decrypt with his private key.
Login to try the interactive encryption demo
LoginHow ECIES Encryption Works
Ephemeral Key Generation
Alice generates a one-time ephemeral key pair for each encryption. This provides forward secrecy - if Alice's main key is compromised, past messages remain secure.
ECDH Key Exchange
Alice uses her ephemeral private key and Bob's public key to compute a shared secret via Elliptic Curve Diffie-Hellman. Only Alice and Bob can derive this shared secret.
HKDF Key Derivation
The shared secret is passed through HKDF-SHA256 to derive a symmetric AES-256 key and nonce. This stretches the shared secret into cryptographically strong key material.
AES-256-CTR Encryption
Data is encrypted using AES-256 in CTR mode. CTR mode is a stream cipher that preserves plaintext length - critical for FlatBuffers where field sizes must remain constant.
Header Transmission
The ephemeral public key and nonce are sent as a header alongside the ciphertext. Bob uses this header plus his private key to derive the same symmetric key and decrypt.
Supported Elliptic Curves
| Curve | Public Key Size | Use Cases | Notes |
|---|---|---|---|
X25519 |
32 bytes | General purpose, modern apps | Fastest, constant-time, recommended for new projects |
secp256k1 |
33 bytes (compressed) | Bitcoin, Ethereum, blockchain | Same keys can derive blockchain addresses |
P-256 / secp256r1 |
33 bytes (compressed) | TLS, NIST compliance, enterprise | Required for government/regulated industries |
P-384 / secp384r1 |
49 bytes (compressed) | High-security, government Top Secret | 192-bit security level, NSA Suite B approved |
Field-Level Encryption
Select which fields to encrypt, generate a record, and visualize the encryption
Select Fields to Encrypt
Check the fields you want to encrypt. Unselected fields will remain in plaintext.
Schema Viewer
View FlatBuffers schema (.fbs) and equivalent JSON Schema
.fbs Schema
JSON Schema
Convert JSON ↔ FlatBuffer
JSON Input
Hex Input (FlatBuffer)
FlatBuffer Studio
Create schemas, generate code, and build FlatBuffers in the browser
Files
Schema Definition
Parsed Structure
Item Editor
Preview
// Schema preview will appear here...
Generated Files
Generated Code
// Generated code will appear here...
Data Entry
Buffer Output
Decoded JSON
Build or upload a buffer
Encryption Module (WASM)
AES-256, X25519, secp256k1, P-256, Ed25519
npm install flatc-wasm
View on npm
Streaming Dispatcher
Route mixed FlatBuffer messages to queues by type
How Streaming Works
Message Flow
FlatBuffers are self-describing binary messages with a 4-byte type identifier prefix. The streaming dispatcher reads this identifier and routes each message to the appropriate queue based on its type.
[TYPE_ID:4bytes][SIZE:4bytes][FLATBUFFER:N bytes]
Message Queues
Each message type has a dedicated queue with a fixed capacity. When full, oldest messages are overwritten (FIFO). This enables constant-memory streaming for high-throughput scenarios without allocations.
- O(1) insertion and removal
- Zero allocation after initialization
- Lock-free single-producer design
Use Cases
- Telemetry - Route sensor data by device type
- Gaming - Separate entity updates from events
- Finance - Dispatch orders, trades, and quotes
- IoT - Process heterogeneous device streams
API Reference
Wire Format
Each message is size-prefixed (little-endian) followed by a 4-byte file identifier for routing, then the FlatBuffer payload.
Memory Layout
Messages are stored in contiguous ring buffers in WASM linear memory. Each registered type gets its own buffer:
| Component | Location | Access Method |
|---|---|---|
| Ring Buffer Base | bufferPtr from registration |
getTypeInfo(fileId).bufferPtr |
| Message N | bufferPtr + (N * messageSize) |
getMessage(fileId, N) |
| Latest Message | bufferPtr + (head * messageSize) |
getLatestMessage(fileId) |
| Raw Memory View | HEAPU8.subarray(ptr, ptr + size) |
new Uint8Array(wasm.HEAPU8.buffer, ptr, size) |
getMessage() returns a Uint8Array view directly into WASM memory - no data is copied.
This view is valid until the next pushBytes() call that overwrites the ring buffer slot.
Initialization
Streaming Data
Access Patterns
Interactive Demo
Aligned Binary Format
Zero-overhead, fixed-size structs from FlatBuffers schemas for WASM/native interop
Why Aligned Format?
Zero-Copy Access
TypedArray views directly into WASM linear memory. No deserialization overhead - just cast and access.
Fixed-Size Layout
Predictable memory layout with proper alignment. Perfect for arrays of structs in shared memory.
FlatBuffers Schema
Use familiar FlatBuffers .fbs syntax. Structs and tables with scalars and fixed-size arrays are supported.
Multi-Language Output
Generate C++ headers, TypeScript classes, and JavaScript modules from a single schema definition.
Code Generator
Paste a FlatBuffers schema below to generate aligned binary format code:
Schema (FBS)
Generated Code
// Generated code will appear here...
Supported Features
Supported
- Scalars: bool, byte, ubyte, short, ushort, int, uint, long, ulong, float, double
- Fixed-size arrays:
[float:3],[int:16] - Hex array sizes:
[ubyte:0x100](256 bytes) - Nested structs (inlined by value)
- Enums with explicit base type
- Fixed-length strings (set String Length > 0)
Not Supported
- Variable-length strings (without String Length)
- Variable-length vectors
- Unions
- Tables with optional fields (use structs)
Usage Example
// C++ - Direct struct access in WASM
#include "aligned_types.h"
void processEntities(Entity* entities, size_t count) {
for (size_t i = 0; i < count; i++) {
Entity& e = entities[i];
e.health -= 10;
e.position.x += e.velocity.x * dt;
// Zero overhead - direct memory access
}
}
// Export for JS binding
extern "C" void update_entities(Entity* ptr, int count) {
processEntities(ptr, count);
}
WASM Interop Patterns
Since aligned structs have no embedded length metadata, array bounds must be communicated out-of-band. Here are common patterns for sharing data between WASM modules:
// C++ WASM - Export pointer and count separately
static Cartesian3 positions[10000];
static uint32_t count = 0;
extern "C" {
Cartesian3* get_positions() { return positions; }
uint32_t get_count() { return count; }
}
// TypeScript - Read using exported functions
const ptr = wasm.exports.get_positions();
const count = wasm.exports.get_count();
const positions = Cartesian3ArrayView.fromMemory(wasm.memory, ptr, count);
for (const pos of positions) {
console.log(`(${pos.x}, ${pos.y}, ${pos.z})`);
}
// Schema - Store indices instead of pointers
struct Cartesian3 { x: double; y: double; z: double; }
table Satellite {
norad_id: uint32;
position_index: uint32; // Index into positions array
velocity_index: uint32; // Index into velocities array
}
// TypeScript - Compute offset from index
const CARTESIAN3_SIZE = 24; // 3 doubles × 8 bytes
class SpaceData {
constructor(memory, positionsBase, satellitesBase) {
this.memory = memory;
this.positionsBase = positionsBase;
this.satellitesBase = satellitesBase;
}
// O(1) lookup by index
getPositionByIndex(index) {
const offset = this.positionsBase + index * CARTESIAN3_SIZE;
return Cartesian3View.fromMemory(this.memory, offset);
}
// Follow cross-reference from satellite to position
getSatellitePosition(satIndex) {
const sat = this.getSatelliteByIndex(satIndex);
return this.getPositionByIndex(sat.position_index);
}
}
// Schema - Manifest with indices into data array
table EphemerisManifest {
satellite_ids: [uint32:100];
start_indices: [uint32:100]; // Where each satellite's data starts
point_counts: [uint32:100]; // How many points per satellite
total_satellites: uint32;
}
struct EphemerisPoint {
julian_date: double;
x: double; y: double; z: double;
vx: double; vy: double; vz: double;
}
// TypeScript - Navigate using manifest
const POINT_SIZE = 56; // 7 doubles × 8 bytes
class EphemerisReader {
constructor(memory, manifestPtr, pointsPtr) {
this.manifest = EphemerisManifestView.fromMemory(memory, manifestPtr);
this.pointsBase = pointsPtr;
this.memory = memory;
}
// Get all points for a satellite
getSatellitePoints(satIndex) {
const startIdx = this.manifest.start_indices[satIndex];
const count = this.manifest.point_counts[satIndex];
const offset = this.pointsBase + startIdx * POINT_SIZE;
return new EphemerisPointArrayView(this.memory.buffer, offset, count);
}
// Get specific point: base + (startIndex + timeIndex) * size
getPoint(satIndex, timeIndex) {
const startIdx = this.manifest.start_indices[satIndex];
const offset = this.pointsBase + (startIdx + timeIndex) * POINT_SIZE;
return EphemerisPointView.fromMemory(this.memory, offset);
}
}
byte_offset = base_ptr + index × STRUCT_SIZE — Since struct sizes are fixed and known at compile time, any index can be converted to a byte offset with a single multiplication.
WASM Runtime Integration
Run the encryption module in any language with WebAssembly support
Why WASM Runtimes?
Single Auditable Implementation
One C++/Crypto++ codebase compiled to WASM. Audit once, deploy everywhere.
Battle-Tested Crypto
Crypto++ has 30+ years of production use in security-critical applications.
Cross-Language Interop
Data encrypted in Go can be decrypted in Python, Rust, Java, and vice versa.
Installation
Install via npm to get the FlatBuffers compiler with encryption support:
npm install flatc-wasm
Works in Node.js, browsers, and any JavaScript runtime with WASM support.
Language Bindings
Example integrations for different languages and runtimes:
Pure Go WebAssembly runtime - no CGo required
go get github.com/tetratelabs/wazero
Run WASM modules with wasmer-python or wasmtime
pip install wasmer wasmer-compiler-cranelift
Pure Java WASM interpreter - no JNI required
com.dylibso.chicory:runtime:1.5.3
Streaming API Examples
FlatBuffers supports streaming multiple messages in a single buffer. Here's how to read them in each language:
import { ByteBuffer } from 'flatbuffers';
import { Monster } from './monster_generated';
// Assume `buffer` contains multiple size-prefixed FlatBuffer messages
const data = new Uint8Array(buffer);
const messages: Monster[] = [];
let offset = 0;
// Read entire buffer of messages
while (offset < data.length) {
const size = new DataView(data.buffer, offset).getUint32(0, true);
const bb = new ByteBuffer(data.slice(offset + 4, offset + 4 + size));
messages.push(Monster.getRootAsMonster(bb));
offset += 4 + size;
}
// Read latest message (last in buffer)
function getLatestMessage(data: Uint8Array): Monster {
let offset = 0;
let lastOffset = 0;
while (offset < data.length) {
lastOffset = offset;
const size = new DataView(data.buffer, offset).getUint32(0, true);
offset += 4 + size;
}
const size = new DataView(data.buffer, lastOffset).getUint32(0, true);
const bb = new ByteBuffer(data.slice(lastOffset + 4, lastOffset + 4 + size));
return Monster.getRootAsMonster(bb);
}
// Read arbitrary indexed message
function getMessageAtIndex(data: Uint8Array, index: number): Monster | null {
let offset = 0;
let currentIndex = 0;
while (offset < data.length) {
const size = new DataView(data.buffer, offset).getUint32(0, true);
if (currentIndex === index) {
const bb = new ByteBuffer(data.slice(offset + 4, offset + 4 + size));
return Monster.getRootAsMonster(bb);
}
offset += 4 + size;
currentIndex++;
}
return null;
}
import (
"encoding/binary"
flatbuffers "github.com/google/flatbuffers/go"
"myapp/Monster"
)
// Read entire buffer of messages
func ReadAllMessages(data []byte) []*Monster.Monster {
var messages []*Monster.Monster
offset := 0
for offset < len(data) {
size := binary.LittleEndian.Uint32(data[offset:])
msg := Monster.GetRootAsMonster(data[offset+4:offset+4+int(size)], 0)
messages = append(messages, msg)
offset += 4 + int(size)
}
return messages
}
// Read latest message
func GetLatestMessage(data []byte) *Monster.Monster {
offset := 0
lastOffset := 0
for offset < len(data) {
lastOffset = offset
size := binary.LittleEndian.Uint32(data[offset:])
offset += 4 + int(size)
}
size := binary.LittleEndian.Uint32(data[lastOffset:])
return Monster.GetRootAsMonster(data[lastOffset+4:lastOffset+4+int(size)], 0)
}
// Read arbitrary indexed message
func GetMessageAtIndex(data []byte, index int) *Monster.Monster {
offset := 0
currentIndex := 0
for offset < len(data) {
size := binary.LittleEndian.Uint32(data[offset:])
if currentIndex == index {
return Monster.GetRootAsMonster(data[offset+4:offset+4+int(size)], 0)
}
offset += 4 + int(size)
currentIndex++
}
return nil
}
import struct
from Monster import Monster
def read_all_messages(data: bytes) -> list:
"""Read entire buffer of messages."""
messages = []
offset = 0
while offset < len(data):
size = struct.unpack('<I', data[offset:offset+4])[0]
msg = Monster.Monster.GetRootAs(data[offset+4:offset+4+size], 0)
messages.append(msg)
offset += 4 + size
return messages
def get_latest_message(data: bytes):
"""Read the latest (last) message in the buffer."""
offset = 0
last_offset = 0
while offset < len(data):
last_offset = offset
size = struct.unpack('<I', data[offset:offset+4])[0]
offset += 4 + size
size = struct.unpack('<I', data[last_offset:last_offset+4])[0]
return Monster.Monster.GetRootAs(data[last_offset+4:last_offset+4+size], 0)
def get_message_at_index(data: bytes, index: int):
"""Read arbitrary indexed message."""
offset = 0
current_index = 0
while offset < len(data):
size = struct.unpack('<I', data[offset:offset+4])[0]
if current_index == index:
return Monster.Monster.GetRootAs(data[offset+4:offset+4+size], 0)
offset += 4 + size
current_index += 1
return None
use flatbuffers;
use crate::monster_generated::Monster;
/// Read entire buffer of messages
fn read_all_messages(data: &[u8]) -> Vec<Monster> {
let mut messages = Vec::new();
let mut offset = 0;
while offset < data.len() {
let size = u32::from_le_bytes(data[offset..offset+4].try_into().unwrap()) as usize;
let msg = flatbuffers::root::<Monster>(&data[offset+4..offset+4+size]).unwrap();
messages.push(msg);
offset += 4 + size;
}
messages
}
/// Read latest message
fn get_latest_message(data: &[u8]) -> Option<Monster> {
let mut offset = 0;
let mut last_offset = 0;
while offset < data.len() {
last_offset = offset;
let size = u32::from_le_bytes(data[offset..offset+4].try_into().unwrap()) as usize;
offset += 4 + size;
}
let size = u32::from_le_bytes(data[last_offset..last_offset+4].try_into().unwrap()) as usize;
flatbuffers::root::<Monster>(&data[last_offset+4..last_offset+4+size]).ok()
}
/// Read arbitrary indexed message
fn get_message_at_index(data: &[u8], index: usize) -> Option<Monster> {
let mut offset = 0;
let mut current_index = 0;
while offset < data.len() {
let size = u32::from_le_bytes(data[offset..offset+4].try_into().unwrap()) as usize;
if current_index == index {
return flatbuffers::root::<Monster>(&data[offset+4..offset+4+size]).ok();
}
offset += 4 + size;
current_index += 1;
}
None
}
import java.nio.ByteBuffer;
import java.nio.ByteOrder;
import java.util.ArrayList;
import java.util.List;
import Monster;
public class StreamingReader {
// Read entire buffer of messages
public static List<Monster> readAllMessages(byte[] data) {
List<Monster> messages = new ArrayList<>();
int offset = 0;
while (offset < data.length) {
int size = ByteBuffer.wrap(data, offset, 4)
.order(ByteOrder.LITTLE_ENDIAN).getInt();
ByteBuffer bb = ByteBuffer.wrap(data, offset + 4, size);
messages.add(Monster.getRootAsMonster(bb));
offset += 4 + size;
}
return messages;
}
// Read latest message
public static Monster getLatestMessage(byte[] data) {
int offset = 0;
int lastOffset = 0;
while (offset < data.length) {
lastOffset = offset;
int size = ByteBuffer.wrap(data, offset, 4)
.order(ByteOrder.LITTLE_ENDIAN).getInt();
offset += 4 + size;
}
int size = ByteBuffer.wrap(data, lastOffset, 4)
.order(ByteOrder.LITTLE_ENDIAN).getInt();
ByteBuffer bb = ByteBuffer.wrap(data, lastOffset + 4, size);
return Monster.getRootAsMonster(bb);
}
// Read arbitrary indexed message
public static Monster getMessageAtIndex(byte[] data, int index) {
int offset = 0;
int currentIndex = 0;
while (offset < data.length) {
int size = ByteBuffer.wrap(data, offset, 4)
.order(ByteOrder.LITTLE_ENDIAN).getInt();
if (currentIndex == index) {
ByteBuffer bb = ByteBuffer.wrap(data, offset + 4, size);
return Monster.getRootAsMonster(bb);
}
offset += 4 + size;
currentIndex++;
}
return null;
}
}
using FlatBuffers;
using System;
using System.Collections.Generic;
public static class StreamingReader
{
// Read entire buffer of messages
public static List<Monster> ReadAllMessages(byte[] data)
{
var messages = new List<Monster>();
int offset = 0;
while (offset < data.Length)
{
int size = BitConverter.ToInt32(data, offset);
var bb = new ByteBuffer(data, offset + 4);
messages.Add(Monster.GetRootAsMonster(bb));
offset += 4 + size;
}
return messages;
}
// Read latest message
public static Monster GetLatestMessage(byte[] data)
{
int offset = 0;
int lastOffset = 0;
while (offset < data.Length)
{
lastOffset = offset;
int size = BitConverter.ToInt32(data, offset);
offset += 4 + size;
}
int lastSize = BitConverter.ToInt32(data, lastOffset);
var bb = new ByteBuffer(data, lastOffset + 4);
return Monster.GetRootAsMonster(bb);
}
// Read arbitrary indexed message
public static Monster? GetMessageAtIndex(byte[] data, int index)
{
int offset = 0;
int currentIndex = 0;
while (offset < data.Length)
{
int size = BitConverter.ToInt32(data, offset);
if (currentIndex == index)
{
var bb = new ByteBuffer(data, offset + 4);
return Monster.GetRootAsMonster(bb);
}
offset += 4 + size;
currentIndex++;
}
return null;
}
}
import FlatBuffers
class StreamingReader {
// Read entire buffer of messages
static func readAllMessages(data: Data) -> [Monster] {
var messages: [Monster] = []
var offset = 0
while offset < data.count {
let size = data.withUnsafeBytes { ptr in
ptr.load(fromByteOffset: offset, as: UInt32.self).littleEndian
}
let slice = data[offset+4..<offset+4+Int(size)]
var bb = ByteBuffer(data: slice)
messages.append(Monster.getRootAsMonster(bb: &bb))
offset += 4 + Int(size)
}
return messages
}
// Read latest message
static func getLatestMessage(data: Data) -> Monster? {
var offset = 0
var lastOffset = 0
while offset < data.count {
lastOffset = offset
let size = data.withUnsafeBytes { ptr in
ptr.load(fromByteOffset: offset, as: UInt32.self).littleEndian
}
offset += 4 + Int(size)
}
let size = data.withUnsafeBytes { ptr in
ptr.load(fromByteOffset: lastOffset, as: UInt32.self).littleEndian
}
let slice = data[lastOffset+4..<lastOffset+4+Int(size)]
var bb = ByteBuffer(data: slice)
return Monster.getRootAsMonster(bb: &bb)
}
// Read arbitrary indexed message
static func getMessageAtIndex(data: Data, index: Int) -> Monster? {
var offset = 0
var currentIndex = 0
while offset < data.count {
let size = data.withUnsafeBytes { ptr in
ptr.load(fromByteOffset: offset, as: UInt32.self).littleEndian
}
if currentIndex == index {
let slice = data[offset+4..<offset+4+Int(size)]
var bb = ByteBuffer(data: slice)
return Monster.getRootAsMonster(bb: &bb)
}
offset += 4 + Int(size)
currentIndex += 1
}
return nil
}
}
FinishSizePrefixed when building to create streamable buffers.
Core WASM Module
The npm package includes everything you need:
flatc-encryption.wasm
Core encryption module compiled from Crypto++ (~1.2MB)
encryption.mjs
JavaScript loader with TypeScript definitions